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ABSTRACT 

*  A  new  explicit  bound  is  given  for  the  ratio  of  the  absolute  error  in  an 
approximate  solution  of  a  system  of  linear  inequalities  and  equalities  to  the 
absolute  residual.  This  bound  generalizes  the  concept  of  a  norm  of  the 
inverse  of  a  nonsingular  matrix.  With  this  bound  a  condition  number  is 
defined  for  a  system  of  linear  inequalities  and  equalities  and  for  linear 
programs.  The  condition  number  gives  a  bound  on  the  ratio  of  the  relative 
error  of  an  approximate  solution  to  the  relative  residual.  In  the  case  of  a 
strongly  stable  system  of  linear  inequalities  and  equalities  the  condition 
number  can  be  computed  by  means  of  a  single  linear  program. 
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SIGNIFICANCE  AND  EXPLANATION 


In  solving  a  system  of  nonsingular  linear  equations  the  condition  number 
gives  a  useful  bound  on  the  ratio  of  the  relative  error  of  an  approximate 
solution  to  the  relative  residual.  We  extend  this  bound  to  the  important 
cases  of  linear  equations  and  inequalities  and  of  linear  programs  which  one 
commonly  encounters  in  operations  research. 
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A  CONDITION  NUMBER  FOR  LINEAR  INEQUALITIES 
AND  LINEAR  PROGRAMS 


0.  L.  MANGASARIAN 


1.  Introduction 

A  well  known  result  of  linear  algebra  [1 ,  8]  states  that  for  the  system 


of  linear  equation 


Ex  ■  g 


where  E  is  a  given  n  x  n  real  nonsingular  matrix  and  g  is  a  given  nonzero 
vector  in  the  n-dimensional  real  Euclidean  space  Rn,  the  following  bound  for 
the  ratio  of  the  relative  error  of  an  approximate  solution  to  the  relative 
residual  holds  for  any  fixed  normt 

Ji-ll!  <  -  3-  .  (2) 

hi  ,s' 

Here  x  is  the  exact  solution,  that  is  Ex  -  g,  whereas  x  is  an 
approximate  solution  with  residual  Ex  -  g.  The  quantity  IE  1 1  IE  I,  which 
depends  on  the  norm  employed,  is  called  the  condition  number  of  the  matrix 
E.  The  condition  number  is  a  useful  numerical  constant  which  measures  how 
badly  the  relative  error  could  behave  in  terms  of  the  relative  residual.  The 
purpose  of  this  work  is  to  define  a  similar  quantity  for  the  system  of  linear 
inequalities  and  equalities 


Ax  <  b 


Cx  -  d 


where  A  and  C  are  given  m  x  n  and  k  x  n  real  matrices  respectively, 
b  is  a  given  vector  in  the  m-dimensional  real  Euclidean  space  R?“  and  d  is 

lr 

a  given  vector  in  R  . 
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Our  results  depend  on  a  theorem  due  to  Hoffman  [4]  which  gives  a  bound  on 
the  ratio  of  the  absolute  error  in  an  approximate  solution  of  a  system  of 
linear  inequalities  to  the  absolute  value  of  the  residual*  We  shall  give  here 
a  new  explicit  expression  for  this  bound  by  using  a  particularly  simple  proof 
that  makes  use  of  duality  theory  of  linear  programming  [3]*  The  bound  in 
Hoffman' s  theorem  is  a  generalization  of  the  idea  of  a  norm  of  the  inverse  of 
a  nonsingular  matrix  (Remark  2).  For  monotonic  norms  the  product  of  this 
bound  with  the  norm  of  the  matrices  involved  generates  what  we  have  termed  a 
condition  number  for  matrices  associated  with  a  system  of  inequalities  and 
equalities  (Theorem  2  and  Definition  1).  For  the  case  of  a  system  of 
inequalities  which  satisfy  a  strong  regularity  condition  we  show  how  the 
condition  number  can  be  obtained  by  solving  a  single  linear  program  (Theorem 
3)  •  In  Section  3  we  extend  the  concept  of  a  condition  number  to  a  pair  of 
dual  linear  programs. 


We  briefly  describe  now  the  notation  and  some  of  the  basic  concepts  used 
in  this  work.  For  a  vector  x  in  the  n-dimensional  real  Euclidean  space  Rn, 
| x |  and  x+  will  denote  the  vectors  in  Rn  with  components  | x | ^  -  |x^| 


and  (x+)i  *  max{x^,0},  i  =  1,2, ...,n,  respectively.  For  a  norm  Ixl^  on 

Rn,  lx  II  #  will  denote  the  dual  norm  [5,  10]  on  Rn,  that  is 
3  T 

lx  I  #  *  max  x  y,  where  the  superscript  T  denotes  the  transpose.  The 

6  lylp-1 

T 

generalized  Cauchy-Schwarz  inequality  |x  y|  <  lx  I  lyl  #,  for  x  and  y  in 

6  0 

Rn,  follows  immediately  from  this  definition  of  the  dual  norm.  For  1  <  p. 


"I  1  * 

q  <  «  and  —  +  —  «  1  the  p-norm  (  T  |xJP)P  and  the  q-norm  are  dual  norms 
P  *  i-1  1 

in  Rn  [8].  If  I  *  H  is  a  norm  on  Rn,  we  shall,  with  a  slight  abuse  of 

P 

notation,  let  1*1  also  denote  the  corresponding  norm  on  R®  for  m  »<n. 

P 


For  an  m  x  n  real  matrix  A,  A^  denotes  the  ith  row  and  A.j 


denotes  the 


»!wwimn 


jth  column,  while  IAI  denotes  the  matrix  norm  [11}  subordinate  to  the 

P 

vector  norm  I  •  I  ,  that  is  IA I  *  max  lAx  I .  .  The  consistency  condition 
B  6  lx  1^-1  B 

KAxlp  <  IA  I  g  lx  lg  follows  immediately  from  this  definition  of  a  matrix 
norm.  A  vector  of  ones  in  any  real  Euclidean  space  will  be  denoted  by  e. 


talities 


I 

I 

! 

i 


2.  Condition  Number  for  a  System  of  Inequalities  and  Bqu 

We  begin  by  stating  and  proving  a  form  of  Hoffman's  theorem  [4]  which  is 
suitable  for  our  purposes  and  which  includes  an  explicit  expression,  not  given 
before,  for  the  value  of  the  bound  on  the  ratio  of  the  error  in  an  approximate 
solution  to  the  residual.  Robinson  [9]  has  given  a  different  but  more 
complicated  expression  for  this  bound. 

Theorem  1.  Let  A,  b,  C  and  d  be  as  defined  in  (3),  let  l»lg  be  a  norm 
on  r!“+’c  and  let 

S  *  {x  1  Ax  <  b,  Cx  *  d}  (4) 

be  nonempty.  Then  for  each  x  in  Rn  not  in  S  there  exists  a  point  p(x) 
in  S  such  that 

*-pU>l„<  pgJ(A,C)  •  K Ax-b)  +,  (Cx-d)lg  (5) 

where 


r 

(u,v,w,y,0  is  a  vertex  of^ 

maximum  < 

*w.  y-ec*  * 

T  T  T 

u-v-A  w  -  C  y+C  ej;  =  0 

T  .  T 

u,v,w,y,c 

e 

e  u  +  e  v  -I 

(u,v,w,y,  c)  >  0  > 

(6) 


Remark  1 . 

The  subscripts  8  and  •  of  jj  refer  to  the  8-norm  and  «»-norm  used 

in  (5).  We  shall  give  below  in  (9)  a  relation  using  other  norms.  The 

condition  in  (6)  that  (u,v,w,y, £)  be  a  vertex  is  essential,  for  otherwise 

the  maximum  of  (6)  may  not  exist.  We  shall  give  below  (Corollary  2  and 

Theorem  3)  cases  where  the  vertex  requirement  need  not  be  imposed. 

Proof  of  Theorem  1.  Let  x  be  any  point  in  Rn  not  in  S.  Because  the  set 

S  is  nonempty  there  exists  a  point  p(x)  in  S  which  is  closest  to  x  in 

the  »-norm.  For  such  a  point,  (p(x),  lp(x)-xl  )  constitute  a  solution  of 

00 


the  linear  program 

min {6  |  -e{  <  p-x  <  e 6,  Ap  <  b,  Cp  •  d}  .  (7) 

Pi  6 

The  dual  of  (7)  is 

max  {xT (u*v)-bT «*dT (y-e{) |u-v-ATw-CTy+CTec  *  0,  eT(u+v)  *  1,  (8) 

u,v,w,y,5  (u,v,w,y,c)  >  0} 

with  a  vertex  solution  (u(x),  v(x),  w(x),  y(x),  g(x))  which  satisfies 

0  <  6(x):-  lp(x)  -  xl  -  xT(u(x)  -  v(x))  -  bTw(x)  -  dT(y(x)  -  ec(x)) 

00 

■  w(x)T(Ax-b)  +  (y(x)  -  e((x) )T(Cx-d) 

<  w(x)T(Ax-b)+  +  (y(x)  -  e;(x)  )T(Cx-d) 

(Since  ( Ax-b)+  >  Ax-b) 

<  lw(  x)  /  (y(x)  -  «5(x))l  #  •  l(Ax-b)  ,  (Cx-d)  I 

(By  the  generalized  Cauchy-Schwarz  inequality) 

<  y  _(A,C)  l(Ax-b)  ,  (Cx-d)  I 
*  8®  +  8 

□ 


Remark  2«  For  the  case  when  Ax  <  b  is  absent  from  (3)  and  (4),  C  is  a 
nonsingular  n  x  n  matrix  and  B  ■  «v  definition  (6)  degenerates  to 

“—(♦»C)  "  ,c_1 


-5- 


Proof  of  Remark  2 


Vi  ( 4>fC)  -  maximum 
3,3  u,v,y,c 


■  maximum 
u,v,y, c 


*  maximum 
u,v 


-  maximum 
1<i<n 


Theorem  1  can  be 


ly-ecl 


1 


(u#v,y, c)  is  a  vertex  oft 
T  _1 

y-e;  -  (C  )  (u-v) 

T  T  . 

e  u  +  e  v  ■  1 

(u,v,y,  z)  >  0 

* 

's 

T  _1 

y-ec-  (C  )  (u-v) 

T  ,  T  . 

e  u  +  e  v  «  1 

(u,v,y,?)  >  0 


(Because  the  convex  function  ly-ecl^  attains 
its  maximum  at  a  vertex  [10,  Corollary  32.3.4]) 


T  _1 

1(C  )  (U-V)  I, 


T  .  T 
e  u  +  e  v 


(u,v)  >  0 


l±  ((cT)  )#1lt 


T 

(Since  I(C  )  (u-v)  I  is  convex  and  attains  its 

maximum  at  one  of  the  vertices  of  the  simplex 
T  T 

{(u,v)|e  u  +  e  v  -  1,  (u,v)  >  0).  These  are  the 
vertices  of  the  unit  cube  {(u,v)|0  <  (u,v)  <  e) 
lying  along  the  coordinate  axes  and  excluding 
the  origin.) 


easily  stated  for  norms  other  than  the  «-norm  as 


follows 


^  a 


s 

L'f 


•V*  im&0e 


Corollary  1 «  Let  the  assumptions  of  Theorem  1  hold*  Then  for  each  x  not  in 
S  there  exists  p(x)  in  S  such  that 

lx  -  p( x)  I  <  u0  (A,C)  •  l(Ax-b)  ,  (Cx-d)l  (9) 

where 


(A,C)  -  a  u„  (A,C) 
By  y—  b* 


(10) 


and  a  is  the  positive  number  relating  the  ynorm  and  »-norm  by 


Izl  <  a  Izl  for  all  z  in  Rn. 

y  s  yoo  oo 


Before  deriving  the  condi tion-number  result  for  the  system  (3)  we  need  a 
couple  of  simple  lemmas*  We  recall  [5*  8]  that  a  monotonic  norm  on  Rn  is 
any  norm  I*  I  on  vP  such  that  for  a,  b  in  tf1,  la  I  <  lb  I  whenever 


1 

a|  <  | b |  .  The  q-norm,  lal  *  (  £  la^l*1)*1  for  q  >  1  is  a 


q 


monotonic  norm 


i=*1 


[8] 


Lemma  1.  Let  a  and  b  be  real  numbers*  Then  a  <  b  implies  (a)+  <  |b| 
Proof.  Let  a  <  b.  Then 

—  I  ■  sx 

I b |  -  (a)  +  -  2(b)  +  -  b  -{a)  +  -  ((b)+  -  b)  +  ((b)  +  -  (a)  +  )  >  0  . 


Lemma  2.  Let  l*l„  be  a  monotonic  norm  on  Rfn+lc,  let  a,  b  be  in  nf"  and 
— — —  fj 


let  c,  d  be  in  R  .  Then  a  <  b  and  c  *  d  imply  that 


Ka)+,  clg  <  lb, dlg. 


Proof.  Let  a  <  b  and  c  =  d.  By  Lemma  1  we  have  that  (a)+  <  |b|,  and  we 


also  have  that  |c|  -  |d|.  Hence  by  the  monotonicity  of  1*1  we  have  that 

P 


Ka)+,  clg  <  lb, dig. 


Lemma  3.  Let  A,  b,  C  and  d  be  as  defined  in  (3)  and  let  l«l  be  a 


raonotonic  norm  on  R®4-*.  Then  l(-b)  +  ,  dig  <  lAx,  Cxlg  for  any  x  in  S. 


Proof .  Just  apply  Lemma  2  to  -b  <  -Ax  and  d  *  Cx. 


□ 


-7- 


1 


4 

♦ 

* 


We  are  now  ready  to  establish  our  condition-number  result  for  the  system 


Theorem  2.  Let  A,  b,  C,  d  and  S  be  as  defined  in  (3)  and  (4),  let  1*1 

3 

be  a  monotonic  norm  on  bP**  and  let  ((-b)+,  d)  +  0.  Then  there  exists  a 

number  p  (A,C)  defined  by  (10)  and  (6)  such  that  for  each  x  in  not 

pp 

in  S  there  exists  a  p(x)  in  S  such  that 


be  -  p( x )  I 


lp(x)  I 


A  l(Ax-b)+,  (Cx-d) I 
-  ^g(A,C)  C  g  l(-b)  +  ,  d»e 


Proof*  Let  x  be  any  point  in  R11  not  in  S.  By  Corollary  1  there  exists  a 
point  p(x)  in  S  satisfying  (9).  Obviously  p(x)  f  0,  because 
( (-b)+,  d)  ¥  0,  and 

lx  -  p(x)lg  l(-b)  +  ,  dlg  l(Ax-b)+,  (Cx-d)lg 

ip(x)  i  «  W33(A'C)  ip(x)i  i(-b)  .  di‘  E 


l(-b)  +  ,  dl  l(Ax-b)+,  (Cx-d)  I 


lp(x)  | 


K-b)  +  ,  dig 


<  Pg6(A,C) 


IAp(x) ,  Cp(x)  I  l(  Ax-b)  +,  (Cx-d)  I 


K-b)  +  .  dig 


(By  Lemma  3) 


l(Ax-b)  ,  (Cx-d)  I 

^  ”66u'c>  c  8  -J 


It  is  evident  from  (11)  that  the  following  is  an  appropriate  definition 
for  a  condition  number  of  (A,C)  relative  to  the  inequalities  and  equalities 


Definition  1.  Let  A,  b,  C  and  d  be  as  defined  in  (3),  and  let  H*l  be  a 

3 

monotonic  norm  on  R®^,  The  condition  number  v  (A,C)  of  the  matrices 

P 

(A,C)  relative  to  the  system  of  inequalities  and  equalities  of  (3)  is  defined 


-iC'.  fair 


where  uao(A,C)  is  defined  by  (10)  and  (6). 

PC 

In  view  of  Remark  2  we  have  that  v  (  $,C)  -  IC  1 »  1C  I  when  C  is 

ao  oo  oo 

nonsingular  and  A  is  not  present. 

We  note  that  the  computation  of  the  condition  number  v  ( A,C)  of  (12) 

p 

depends  on  computing  y_  (A/C)  of  (6)  which  is  a  problem  of  maximizing  of  a 

P°° 

convex  function  on  the  vertices  of  a  polyhedral  set.  This  is  a  difficult 
problem  because  it  may  have  many  local  maxima.  This  problem  however  can  be 
reduced  to  a  maximization  problem  on  the  entire  polyhedral  set  rather  than  on 
its  vertices/  if  the  objective  function  is  bounded  above  on  the  entire 
polyhedral  set.  In  particular  we  have  the  following? 

Corollary  2.  Let  A  and  C  be  m  x  n  and  k  x  n  real  matrices 
respectively  and  let  !•!  be  a  norm  on  R®**.  If 

p 

{x | Ax  <  0,  Cx  =*  0}  ¥  4>  and  the  rows  of  C  are  linearly  independent  (13) 
then  the  constant  y„  (A,C)  of  (6)  is  also  given  by 

T  T 

(  u-v-A  w  -  C  z  -  0 

WajA,C)  *  maximum  <  lw,zl  #  T  T 
p  u.V/W/Z  \  B  e  u  +  e  v  =  1 

(u,v,w)  >  0 

Proof.  Note  that  problem  (14)  is  obtained  from  (6)  by  dropping  the  vertex 
condition  from  (6)  and  defining  z  »  y-e^.  We  first  show  that  the  feasible 
region  of  (14)  is  bounded  under  assumption  (13).  Suppose  not/  then  there 
exists  a  sequence  {(ui,vi,wi,zi') }  with  {IwSz*!}  +  »  for  some  norm, 
{(u^/V^)}  remaining  bounded  and 


Hence  by  the  Bolzano-Weiers trass  theorem  there  exists  (w,z)  such  that 

A ^  +  CTz  -  0,  w  >  0,  (w,I)  +  0  (15) 

which  contradicts  the  linear  independence  of  the  rows  of  C  if  w  -  0  and 

contradicts  the  nonemptiness  of  (x  |  Ax  <  0,  Cx  ■  0}  if  w  y  0  because  then 
T  T"  T  T“ 

0  >xAw  +  xCz«0.  Hence  the  feasible  region  of  (14)  is  compact  and  the 

continuous  objective  function  lw,zl  #  attains  a  maximum  on  it.  Since 

0 

problem  (14)  is  equivalent  to  problem  (6)  without  the  vertex  requirement,  then 
problem  (6)  without  the  vertex  condition  must  have  a  solution.  Since  the 
objective  function  of  (6)  is  concave  and  the  feasible  region  is  a  polytope  not 
containing  straight  lines  going  to  infinity  in  both  directions,  problem  (6) 
without  the  vertex  condition  must  have  a  vertex  solution  [10,  Corollary 
32.3.4].  Hence  under  assumption  (13)  the  vertex  condition  can  be  dropped  from 
(6)  which  results  in  (14). 

□ 

Remark  3.  It  can  be  shown  [7]  that  condition  (13)  is  equivalent  to  the  strong 
regularity  or  strong  stability  condition  that 

nrt-k 

Ax  <  b,  Cx  «  d  has  solution  x  for  each  (b,d)  6  R  .  (16) 

For  the  case  when  there  are  no  equalities  present  in  (3),  0  -  «®  and 
0  =»  1,  problem  (14)  becomes  a  simple  linear  program  and  the  condition 
number  of  the  matrix  A  with  respect  to  the  inequalities  Ax  <  b  can  be 
determined  by  solving  a  single  linear  program.  In  particular  we  have  the 
following. 

Theorem  3.  Let  A  be  an  ra  x  n  real  matrix,  let  b  be  a  vector  in  Rf1, 
let  S  *  {x | Ax  <  b }  and  let  fct  | Ax  <  0 }  f  0.  Then  for  each  x  in  R™  not 
in  S  there  exists  a  point  p( x)  in  S  such  that 

lx  -  p(x)  I  <  y  (A)  l(  Ax  -  b)  I  (17) 

where 


-10 


W _ (A)  ”  max  <  •  w 


U  ,v,w 


T  T  T 

u-v-A  w  *  0,  e  u  +  e  v  ■  11, 


u,v,w  >  0 


(18) 


Furthermore  if  (-b)+  0 


be  -  p(x)  l( 
lp(x)  I 


l( Ax  -  b)  I 


<  V  (A)  IAI 


+  « 


(19) 


The  number  p^jA)  IA is  termed  as  the  condition  number  of  the  matrix  A 
relative  to  the  inequalities  Ax  <  b. 


11- 


9 


3.  Condition  Humber  of  a  Linear  Program 


We  now  apply  the  results  of  the  previous  section  to  obtain  some 
corresponding  results  for  a  linear  programming  problem.  Consider  the  linear 
program 

Minimize {gT z  |  Hz  >  h,  z  >  0}  (20) 

n 

z6R  1 

n  4  nj 

when  g  and  h  are  given  vectors  in  R  and  R  respectively  and  H  is 
a  given  n2  x  real  matrix.  Associated  with  the  above  linear  program  and 
its  dual  are  the  following  necessary  and  sufficient  optimality  conditions 


C3,  2] 


where 


-Mx  -  q  <  0,  -x  <  0,  q  x  <  0 


(21a) 


M 


0 


H 


(21b) 


We  have  the  following  counterparts  for  the  theorems  of  Section  2. 
Theorem  4.  Consider  the  linear  program  (20)  and  its  associated  optimality 
conditions  (21).  Let 

S  -  {x|-Mx  <  q,  -x  <  0,  qTx  <  0}  f  $  (22) 

and  let  !•!  be  a  norm  on  R°  where  m  »  2(m1  +  n^)  +  1.  Then  for  each 
x  not  in  S  there  exists  a  point  p( x)  in  S  such  that 


lx  -  p(x)  I  < 
00  ■ 


K “Mx  -  q,  -x,  q  x)  I 


(23) 


where 


maximum 

u,v,w1,w2,w3 


lw«#w-#w.» *  * 

B 


(u/v,w1,w2#w3)  is  a"! 

vertex  of: 

T 

u-v+M  w1+w2~qw3  -  0 
T  T 

e  u+e  v  *  1 
(u,v,w  ,w  ,w  )  >  0 


(24) 


If  the  “-norm  is  replaced  by  the  ynorm  in  the  above ,  then 


must  be  replaced  by 


w 


(25) 


where  a  ^  is  the  positive  number  relating  the  y-norms  and  the  “-norms  by 

n.+m- 


Izl  <  o  Izl  for  all  z  in  R 
Y  -  K“  " 


ir“i 


Theorem  5.  (Condition  number  of  a  linear  program)  Let  the  assumptions  of 

2(m1+n1)+1 

Theorem  4  hold,  let  1*1  be  a  monotonic  norm  on  R  and  let 

P 

/ 

-M 

q  £  0.  Then  there  exists  a  number  f  -X  j  defined  by  (25)  and  (24) 


such  that  for  each  x  not  in  S  there  exists  a  p(x)  in  S  such  that 


lx  -  p(  x)  I 


I 


ilp(x)  I.  »  PB8 


I  -m\ 

-M 

-1 

-I 

\  T  / 

T 

\q  / 

q 

(-MX-q,-x,qTx)+j 


(26) 
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The  number 


(27) 


is  defined  as  the  condition  number  of  the  linear  program  (20)* 

We  note  that  because  Mx  +  q  >  0,  x  >  0  imply  that  qTx  >  0,  it  follows 
that  there  exists  no  x  satisfying  Mx  >  0,  x  >  0  and  qTx  <  0  and  hence 
the  problem  (24)  cannot  be  reduced  to  a  linear  program  as  was  done  in  Theorem 
3. 
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